Methods and system for producing a temperature map of a scene

ABSTRACT

A method for generating a temperature map of a scene may include receiving thermal data of the scene, wherein the thermal data includes a plurality of frames of thermal infrared data. A mapping may be created for each frame of the plurality of frames based on the digital thermal infrared data. The method further comprises generating the temperature map using the mapping, wherein the temperature map is generated prior to a contrast enhancement process and separately transmitting the temperature map and the digital thermal infrared data in a data channel.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/206,297, entitled “METHODS AND SYSTEMS FOR PRODUCING A TEMPERATUREMAP OF A SCENE,” filed Mar. 12, 2014, which claims priority to U.S.Provisional Patent Application No. 61/786,077, filed Mar. 14, 2013,entitled “METHOD AND SYSTEMS FOR PRODUCING A TEMPERATURE MAP OF ASCENE,” and U.S. Provisional Patent Application No. 61/785,856, filedMar. 14, 2013 entitled “METHOD AND SYSTEM FOR PROVIDING SCENE DATA IN AVIDEO STREAM,” the disclosures of which are hereby incorporated byreference in their entirety for all purposes. This application isrelated to U.S. Pat. No. 9,251,595 corresponding to U.S. patentapplication Ser. No. 14/211,796, filed on Mar. 14, 2014, entitled“METHOD OF SHUTTERLESS NON-UNIFORMITY CORRECTION FOR INFRARED IMAGERS,”and U.S. application Ser. No. 14/206,341 filed Mar. 12, 2014, entitled“METHOD AND SYSTEM FOR PROVIDING SCENE DATA IN A VIDEO STREAM,” thedisclosure of which is hereby incorporated by reference in its entiretyfor all purposes.

BACKGROUND OF THE INVENTION

The electromagnetic spectrum encompasses radiation from gamma rays,x-rays, ultra violet, a thin region of visible light, infrared,terahertz waves, microwaves, and radio waves, which are all related anddifferentiated in the length of their wave (wavelength). All objects, asa function of their temperatures, emit a certain amount of radiation.For terrestrial objects, a significant portion of this radiation isemitted in the infrared.

Thermal cameras can detect this radiation in a way similar to the way aphotographic camera detects visible light and captures it in aphotograph. Because thermal cameras detect and capture infrared light,thermal cameras can work in complete darkness, as ambient light levelsare not needed. Images from infrared cameras typically have a singlecolor channel because thermal cameras generally use sensors that do notdistinguish different wavelengths of infrared radiation. Color thermalcameras require a more complex construction to differentiate wavelengthand color has less meaning outside of the normal visible spectrumbecause the differing wavelengths do not map uniformly into the systemof color visible to and used by humans.

The monochromatic images from infrared cameras are often displayed inpseudo-color, where changes in color are used, as opposed to changes inintensity, to display changes in the signal, for example, gradients oftemperature. This is useful because although humans have much greaterdynamic range in intensity detection than color overall, the ability tosee fine intensity differences in bright areas is fairly limited.Therefore, for use in temperature measurement, the brightest (warmest)parts of the image are customarily colored white, intermediatetemperatures reds and yellows, transitioning to blues and greens, withthe dimmest (coolest) parts black. A scale should be shown next to afalse color image to relate colors to temperatures.

Thermal cameras have many applications, particularly when light andvisibility are low. For example, thermal cameras have been used inmilitary applications to locate human beings or other warm entities.Warm-blooded animals can also be monitored using thermographic imaging,especially nocturnal animals. Firefighters use thermal imaging to seethrough smoke, find people, and localize hotspots of fires. With thermalimaging, power line maintenance technicians locate overheating jointsand parts, a telltale sign of their failure, to eliminate potentialhazards. Where thermal insulation becomes faulty, building constructiontechnicians can see heat leaks to improve the efficiencies of cooling orheating air-conditioning. Thermal imaging cameras are also installed insome luxury cars to aid the driver at night. Cooled infrared cameras canbe found at major astronomy research telescopes, even those that are notinfrared telescopes.

SUMMARY OF THE INVENTION

Embodiments of the present invention generally relate to optical systemsand methods of processing video signals, for example thermal videosignals. Methods and systems for producing a supportive data map of ascene image are provided. More particularly, the present inventionrelates to systems and methods of processing supportive data such astemperature data for thermal video signals. Merely by way of example,radiometric infrared cameras and systems may be used to gather thermaldata of a scene, and generate thermal video and a temperature map of thescene.

According to an embodiment of the invention, a radiometric thermalcamera is provided. For example, a radiometric thermal camera can be aradiometric infrared camera. The radiometric thermal camera can beconfigured to detect and capture other supportive data (e.g., accoustic,analytic, depth, thermal, X-ray, etc.) of a scene along with baselineimage data of the scene. In other embodiments, various devices may beused to detect and capture visible and non-visible light (e.g., thermaldata). In one embodiment, thermal data may be captured and processedinto a format (e.g., converted from analog to digital) such that atwo-dimensional matrix representing a temperature map of the scene, inaddition to thermal video (i.e., contrast) data, can be generated fromthe thermal data. The temperature map may be generated using a look-uptable (LUT). The thermal video data may be processed separately from thetemperature map. Additionally, the thermal video data and temperaturemap may be transmitted separately to a camera interface for display to auser. One advantage of processing the video data and temperature dataseparately is that it enables a greater amount of flexibility in themethod of contrast enhancement used for display of video data than othermethods of transmitting temperature information.

Video data may include visible light captured by the camera, andwavelengths of visible light captured typically may be in the range of400-700 nm. Thermal data may include non-visible light, e.g. infraredlight, captured by the camera, and wavelengths of infrared lightcaptured may be as long as 14 μm. The video data in one embodiment ofthe invention can come from a thermal sensor. The thermal sensor candetect emitted radiation in a long wave infrared (LWIR) spectrum, andmay not detect reflected radiation in the visible spectrum. The thermalsensor may operate in the LWIR spectrum (nominally 8-14 μm), and mayoutput thermal video. From the post-NUC (pre-AGC) thermal video, a LUTmay be used to create a temperature map. The camera then outputs a datastream that includes both the thermal video and the temperature map.

The range of thermal data can be much larger than the range of videodata. However, a relevant range of thermal data may be much narrowerthan a relevant range of video data. For example, video data of a sceneincluding a person may involve many varying factors, such as color,shadows, light, and movement. Thus, a majority of visible light range isrelevant as it is visible and detectable to the naked eye. However,temperature of a scene typically does not vary as drastically or quicklyas visible light factors. Even when the person in the scene is moving,the body temperature of the person remains within a small range varyingslightly around normal body temperature of 98.6° F. Since the variationsin the temperature are so small, depending on an ambient temperature ofthe scene, a relevant range of temperature may be much narrower than therelevant range of visible light captured in video.

In one embodiment of the invention, the video contrast informationdescribed herein may be captured by a thermal camera. A change in thethermal radiation from an object in the scene may correspondsimultaneously to a change in both video contrast and temperatureinformation. The temperature information may be tracked at the same ordifferent precision and timescale that is important for viewing videocontrast information, and that updated temperature information may beprovided and refreshed to the system less frequently than video contrastinformation.

Accordingly, processing and transmitting video data and thermal data,for example, temperature data, at the same frequency and same range maybe bandwith-intensive, utilizing heavy processing resources, memory, andincreasing processing time. However, other processing techniques ofthermal data may compromise accuracy and completeness of thermal data(e.g., tempertaure data). Embodiments of the invention address this andother problems, and provide many benefits and advantages.

Methods of and systems for generating a temperature map of a scene areprovided. The method includes receiving video data of the scene, whereinthe video data is in analog format and includes a plurality of frames.The video data is converted analog to digital format to create digitalvideo data. Using the digital video data, a lookup table is created foreach frame of the plurality of frames. A temperature map representingtemperature data of the scene is generated using the lookup table. Thetemperature map is then transmitted separately from the digital videodata in the same data channel.

According to embodiments of the invention, different processing andtransmission techniques may be used to separately process and transmitthermal data and video data of a scene. Thermal data may includetemperature data. For example, since temperature data may vary in asmaller relevant range and less quickly as, and may be tracked at thesame or different precision and timescale than video data, temperaturedata may be captured and processed at a rate slower than video data,without significantly compromising accuracy and completeness of thermaldata of the scene, for example the temperature data of the scene.

In one embodiment of the invention, a method of generating a temperaturemap of a scene is provided. The method includes receiving thermal dataof the scene. The thermal data can include a plurality of frames ofthermal infrared data. Further, the temperature map may be generatedbased on at least the thermal data, and the temperature map can begenerated prior to a contrast enhancement process of the thermal data.The method can further comprise transmitting the temperature map and thethermal data concurrently (e.g., simultaneously) in a data channel as adata stream. The temperature map can be generated using a lookup table.As an example, the lookup table can be created by converting videolevels to temperature. In an embodiment, the lookup table is apredetermined lookup table.

In another embodiment of the invention, an apparatus for generating atemperature map of a scene is provided. The apparatus can include adetector configured to receive thermal data of the scene, the thermaldata including a plurality of frames of thermal infrared data. Acalibrated mapper in the apparatus may be configured to create a mappingfor each frame of the plurality of frames based on the digital thermalinfrared data. Additionally, the apparatus may comprise a temperaturemap generator may be configured to generate the temperature map usingthe mapping; and a data channel interface configured to transmit thetemperature map separately from the digital infrared light data.

Furthermore, in other embodiments, the temperature data may be providedat a different spatial resolution as the video data, and temperaturedata can be captured and processed at a lower resolution. Thus, benefitsand advantages achieved by embodiments of the present invention includereduced bandwidth without comprising accuracy and completeness ofthermal data, for example temperature data. These and other embodimentsof the invention, along with many of its advantages and features, aredescribed in more detail in conjunction with the text below and attachedfigures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified schematic diagram illustrating a radiometricthermal camera (e.g., radiometric infrared camera) according to anembodiment of the present invention.

FIG. 2 shows an example block diagram illustrating an example systemaccording to an embodiment of the invention.

FIG. 3 illustrates an example flow diagram of a method according to oneembodiment of the invention.

FIG. 4 illustrates an example block diagram of a system according to oneembodiment of the invention.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Embodiments of the invention are related to methods and systems, such asradiometric thermal cameras (e.g., radiometric infrared cameras),enabled to capture, process, and transmit video and supportive data,such as thermal and temperature data. In an embodiment of the invention,thermal data may be processed into a two-dimensional matrix representinga temperature map of a scene. The temperature map may be generated usinga look-up table, and transmitted separately from the video data. Thetemperature map may be transmitted at a different frame rate than aframe rate at which the video data is transmitted. Furthermore, thetemperature map may be transmitted at a different spatial resolutionthan the resolution at which the video data is transmitted.

FIG. 1 is a simplified schematic diagram illustrating a radiometricthermal camera (e.g., radiometric infrared camera) according to anembodiment of the present invention. The radiometric thermal camera(e.g., radiometric infrared camera) can be a thermal camera having abody 102. Additional mechanical interfaces may include the body 102being enabled to be mounted via a universal consumer device interface ora ¼″-20 Tripod interface. The camera can include several components foruser interfaces in which a user can operate the camera. For example, insome embodiments, the camera can have a lens focus ring 104, which mayinclude a lens with a shutter (internally, not shown), and/or a lens cap106. To turn the camera on and off, there may be a power switch 108.Additionally, there may be an indicator 110 representing the power stateand to indicate when the battery is charging. For example, it may be alight (e.g., LED) that is green when the battery is above a certainthreshold and red when the battery is below the threshold, warning theuser of limited use so that the battery can be charged. In anotherexample, the indicator 110 may be a light that is green when the poweris on and red when the battery is charging.

The thermal camera may also have several electrical interfaces, and maybe configured to have a wireless interface 112, enabling the camera toreceive and transmit communication and data wirelessly. For example, thewireless interface 112 may be enabled for bidirectional communicationand/or video output using Wi-Fi protocols (e.g., 802.11, etc.).Alternatively, the thermal camera may also be configured for wiredcommunication through a USB (Universal Serial Bus) interface 114, a USBMini interface, or the like. The USB interface 114 may be configured totransmit communication 116, bidirectional streaming video out 118,and/or power in 120.

The wireless interface 112 is a bi-directional communications that maybe enabled to transmit the temperature map captured and generated by thecamera. The temperature map may also be transmitted across the streamingvideo 118. Thus, the camera 102 may employ two method of transmittingvideo and data (e.g., temperature map) from the camera. In anembodiment, the wireless interface 112 may be the primary method totransmit the temperature map. The USB interface 114 may be used toprovide power to charge the camera through the power connection 120, forexample, to charge a battery in the camera for mobile use. The USBinterface 114, in another embodiment, may also include a communicationschannel 116 for control signals, for example, for use as a factoryinterface to upload calibration settings or perform testing.Alternatively, the communications channel 116 may be used by an end userto upload control settings, such as custom programming, calibrationcoefficients, etc.

In an embodiment, the wireless interface 112 may be used to transmitvideo and/or data (e.g., a temperature map) to another mobile device,such as a smartphone. For example, an end user having a mobile device(e.g., smartphone, tablet) can connect to the thermal camera via awireless connection (e.g., wi-fi) to receive thermal video and thetemperature map on his mobile device. The mobile device may host andoperate applications to view and/or process the thermal video and data.For example, the mobile application may comprise code enabling variousfunctions with the video data, such as displaying the thermal videodata, viewing the temperature map, using the temperature map to measuretemperatures for radiometric purposes, such as overlay symbology, colorpallets, etc. The end user may also physically connect his mobile deviceto the thermal camera 102 via the USB interface 114, such that via thestreaming video connection 118, the thermal camera 102 can transmitvideo and temperature data to the end user's mobile device.

FIG. 2 illustrates an example functional block diagram of a radiometricthermal camera (e.g., radiometric infrared camera) according to anembodiment of the invention. The radiometric camera may include a lens202 for a detector 204. The camera may also include a focus ring and/orshutter. The lens 202 and detector 204 can be capable of capturinginfrared radiation over a horizontal field of view (HFOV) of 40° and/orcan be enabled for manual focus by a user using the focus ring. In anembodiment of the invention, the detector may utilize a lens or othersuitable device for capturing and receiving infrared light. In otherembodiments of the invention, a suitable detector may be capable ofcapturing and receiving visible light and infrared (e.g., non-visible)light. In other embodiments, the detector and lens are capable ofcapturing light of other spectrums, such as visible or reflectiveinfrared. The detector 204 can be configured to capture data containingthermal data at a predetermined pixel resolution (e.g., 320×240 pixelresolution) at a predetermined frame rate (e.g., 60 Hz). In otherembodiments, other suitable devices may be used to capture datacontaining both visible light and infrared (e.g., thermal) data. Inother embodiments of the invention, other suitable devices may be usedto capture data containing both light and supportive data (e.g.,temperature, acoustic, X-ray, etc.) The received data may then beprocessed by an analog-to-digital converter (ADC) 206.

The digital signal can be sent to a non-uniformity correction (NUC) andbad pixel replacement (BPR) module 208. The output from the NUC and BPRmodule 208 can be sent to a temperature map generator module 216. Thedigital signal from the ADC 206 is processed to perform non-uniformitycorrection and bad pixel replacement in NUC/BPR module 208.Non-uniformity correction attempts to compensate for responsevariability. For example, a scene with a constant temperature willappear as a flat field, therefore ideally the output of the detectorwould be constant. The output of the detector would appear as a samevideo level or gray level for all the pixels in the thermal image. TheNUC/BPR module 208, using BPR algorithms, normalizes the constant videolevel by scanning for bad pixels, which are detected as statisticaloutliers from the rest of the pixels, and substitute the bad pixels withinformation from their adjacent pixels.

Thermal video data from the NUC/BPR module 208 can be processed by aframe integrator 210. The frame integrator 210 basically allows thethermal camera to combine information from different frames that occurin different points in time. Therefore, the plurality of frames receivedare temporally integrated to help reduce temporal noise components. Inan embodiment, a special noise filtering module 211 to filter spatialnoise from the output of the frame integrator 210. Spatial noisefiltering processing can be similar to temporal noise filtering suchthat it detects and filters out spatial noise instead of temporal noisethat is filtered out by the frame integrator 210.

Thermal video data from the NUC/BPR module 208 can be processed by theframe integrator 210, and then have its frame rate adjusted/throttledappropriately by the throttle 212 (e.g., to output 30 Hz or 9 Hz video).The throttle 212 may be used to regulate the frame rate to apredetermined frequency. The detector 204 may receive data at a certainframe rate, for example, at 60 Hz, and the throttle 212 may adjust thatframe rate to a lower frequency, for example, to 9 Hz or 30 Hz. Loweringthe frame rate may be to satisfy certain export requirements and also tohelp manage bandwidth.

In other embodiments of the invention, temperature data may be separatedfrom other video captured by the lens. In other embodiments, thermalvideo data may be separate from other video captured by the lens. Otherframe rates can be utilized as well. The frame integrator 210 combinesthe signal from sequential frames to reduce noise. There can be someframe rate adjustment in the visible video data by the throttle 212. Forexample, video data captured by the detector at 60 Hz can be adjusted to30 Hz or 9 Hz, depending on intended future applications of the videodata. The output of the throttle 212 may then be processed by anAutomatic Gain Control (AGC) module 214.

Typical scenes encompass a fraction of a full dynamic range, so inmapping the entire AGC output to the display, the scenes would appeargray and washed out. To compensate, the AGC tries to take the fractionof that dynamic range, and map the fraction to the output. This mappingof a smaller portion allows visibility of hotter objects in the scenewith the contrast enhanced. For example, though the objects might not beat the AGC limit, higher temperatures are mapped to white pixels. Colorsin the scene are faded, and even though they might not be at the lowestend of the AGC, may be mapped to the black pixels. As a result, the AGC214 may serve to provide contrast enhancement of the temperature of thescene for display to an end user.

The output of the AGC 214 may then be sent to a multiplexer 224, used toselect only one video output at a time. For example, in one mode ofsystem operation, the multiplexer 224 selects the output of the AGCblock 214. The other inputs (e.g., output from A-to-D converter, frameintegrator, and/or throttle) may be only selected for engineering uses.

According to one embodiment of the invention, the analog-to-digitalconverter may be a 14-bit A-to-D converter. Therefore, there may bevalues ranging from zero to 16,383 (2¹⁴) digital values based on whatthe detector is capturing. Detectors may be set up so that they cancapture a very broad range of scene temperatures.

For example, in a scene, it may include objects as cold as −20° C. toobjects as hot as 120° C. Additionally, the camera may have an operatingmode over a wide range of camera temperatures (i.e., ambient temperatureof operation). For example, the radiometric infrared system according toan embodiment of the invention may operate down to 0° C. (i.e., thecamera's ambient environment is actually 0° C.) and up to 50° C.Therefore based on the range of scene temperatures and the differentambient temperatures of operation, detectors may be configured toaccommodate such that the scene temperatures may be captured over thefull ambient temperature range. Accordingly, for most scenes, at anytime, typically there may not be objects as cold as −20° C. and as hotas 120 C. Typical scenes may be at room temperature, in the range of20-30° C. Therefore, although the detector captures a full 14-bit A-to-Drange which was configured to accommodate the full scene temperaturerange and the full ambient temperature range, there may not besignificant contrast in temperature across the entire range. Only asmall range of the 0 to 16,383 values may contain significant variationin temperature data.

The AGC module 214 takes the smaller range of significant data, and itoptimizes the contrast in that range to typically an 8-bit, therefore256 values, output that would then be displayed to users. Therefore,instead of the users seeing a bland gray scene without a lot of contrastvisible to the eye a smaller range that contains the significant data isamplified. The smaller range is amplified to fill a 256-count range sothat users can see the contrast in the scene.

In one embodiment of the invention, the AGC module 214 may serve tomaintain and/or increase radiometric accuracy and temperaturemeasurement. Therefore the detectors may be calibrated to beradiometrically accurate, so that they are calibrated to not only detecthotter objects to be a specific color or a specific shade of gray, andcolder objects to be another color or another shade of gray. To achievethis radiometric accuracy, the non-uniformity correction module 208 maybe used in the calibration of the detector.

The AGC module 214 may use a linear or non-linear process to optimizethe video contrast. The AGC module 214 may be an adaptive system wherean average output signal level is fed back to adjust the gain to anappropriate level for a range of input signal levels. Additionally, theAGC module 214 may also preserve the temperature and radiometricaccuracy of the video going through that block. However, in some AGCmethodologies, it may be difficult to keep track of the temperature datagoing through the AGC block. Other existing methods for keeping track oftemperature information involve the use of a look-up table (LUT) forevery frame of video, which may represent a mapping from the input14-bit values (i.e., 0 to 16,383 input values) to an 8-bit or 256 valuerange output.

In mapping the 16,383 values in the input to 256 values in the output,some variations in the input may be combined such that not allvariations are mapped to the output. However, since a LUT exists for AGCof every frame, the LUT may be used to determine from an 8-bit outputvideo level the original 14-bit input level. Thus, for these types ofcontrast enhancement that adhere to such a mapping, the radiometricaccuracy can be maintained and a mapping from a post-AGC video level toa particular temperature can be determined. As opposed to a LUT that maybe used for AGC, in which the AGC LUT can be updated for each frame ofvideo, the LUT according to one embodiment of the invention used togenerate a temperature map from the post-NUC video can be a fixed LUT.The fixed LUT remains unchanged from each frame of video, ambienttemperature, and/or camera-to-camera. The fixed LUT may be used toconvert the calibrated post-NUC video levels to a temperature togenerate a temperature map. In another embodiment of the invention, theLUT used to generate a temperature map from the post-NUC video can be adynamic LUT.

A variety of contrast enhancement methods can be used in the AGC module214. Non-reversible processes may be implemented, for example, LocalArea Processing (LAP). During LAP of an image, a series of frequencydomain processes and decompositions of the image are performed, contrastmanipulation is performed, and then the image is reconstructed. For suchmethods of contrast enhancement, mapping of video levels to temperaturemay be complex. According to embodiments of the invention, at the outputof the NUC/BPR module 208, a complete temperature map is generated bythe temperature map generator 216 using a LUT. Unlike a LUT that may beused in AGC in which the LUT changes from frame-to-frame, the LUT usedto generate the temperature map can be fixed for all frames. Bycomputing the temperature information at this point, a greater degree offlexibility can be achieved in the methodology selected for contrastenhancement. For example, a camera with a thermal sensor may operate inthe LWIR spectrum (nominally 8-14 μm), and may output thermal video.From the post-NUC (pre-AGC) thermal video, a LUT may be used to create atemperature map. The camera then outputs a data stream that includesboth the thermal video and the temperature map.

In some embodiments of the invention, the NUC/BPR module 208 may beimplemented using a field-programmable gate array (FPGA) integratedcircuit. FPGA's may be programmed after manufacturing, and thus arecapable of being used in many applications. In FIG. 2, for example, thetemperature map generator 216, frame integrator 210, throttle 212, andAGC module 214 may be implemented using an FPGA. Other suitableintegrated circuits may also be used to implement the functions of themodules described above, for example, application-specific integratedcircuits (ASIC). Two-dimensional matrix temperature map can be generatedafter processing by the NUC/BPR module 208. Also, temperature mapgeneration can occur in parallel with AGC processing of video in the AGCmodule 214.

The multiplexor 224 may allow to select video from various points foroutput downstream, for example, the multiplexor 224 may select a rawinput from the AGC 206, post-NUC/BPR data from the NUC/BPR module 208,the output from the frame integrator 210, the output from the spatialnoise filter 211, or post-throttled data from the throttle 212. Havingthe multiplexor 224 to have the ability to select from various outputsallows flexibility in the potential processing functions depending onthe data used.

The select signal for multiplexor 224 may be a serial command to thecamera that may be provided by an end-user. Various outputs may beselected for processing functions for factory calibration and tests. Inanother embodiment, the select interface may be enabled to be used as acustomizable user interface. For example, the user may wish to processthe raw BPR output data from the NUC/BPR module 208 to build acustomized contrast enhancement or filtering.

As an example, a radiometric thermal camera (e.g., radiometric infraredcamera) according to an embodiment of the invention may have aresolution of 320×240 pixels. Thus, in addition to the 320×240 pixelresolution, the 8-bit video level that is outputted from the AGC module214, a 320×240 pixel temperature map is generated by the temperature mapgenerator 216. The temperature map includes data representing atemperature of every pixel in the image of the scene. The temperaturemap, along with an 8-bit video level output from the output ofmultiplexor 224 may be transmitted to a camera serial interface 218.According to an embodiment of the invention, an output from the cameraserial interface 218 may be processed by a converter 220 to convert toYCbCr, for color image pipeline processing. After conversion to YCbCr,an encoder 222 may be used to encode using for example, H.264 hardwareencoding. According to an embodiment of the invention, the converter 220and encoder 222 may be implemented using system-on-a-chip (SoC)integrated circuits. SoC's may process digital, analog, mixed-signal,and often radio-frequency functions on a single chip substrate, and thusmay be used for embedded systems and communication interfaces. Forexample, SoC's may be used to implement wireless interface 228, USBinterface 232, system communications interface 230, and/or Ethernetinterface 226 in FIG. 2.

The camera serial interface 218 may serve to manage communications withthe outside world. The converter 220 and encoder 222 perform videoformatting for the output. The encoder 222 may implement variouscompressing techniques. The Ethernet interface 226 may communicate withthe wireless interface 228 of the camera. Therefore the camera serialinterface 218, converter 220, encoder 222, and Ethernet interface 226receive the thermal data processed by the FPGA (containing the ADC 206,NUC/BPR module 208, frame integrator 210, spatial noise filter 211,throttle 212, AGC module 214, and multiplexor 224) and configuring thedata to be send out over the wireless interface 228. The wirelessinterface 228 can be in parallel with the USB interface 232. The USBinterface 232, as mentioned above, may also be coupled to the powerbattery charger interface 234.

According to an embodiment of the present invention, the radiometricthermal camera (e.g., radiometric infrared camera) may generate a320×240 resolution temperature map representing the temperature of everypixel in a scene as a 16-bit value encoded using an 11.5 encoding schemein degrees Kelvin. In the 11.5 encoding scheme, the 11-bit is theinteger part of the pixel temperature and the 5-bit is the decimalportion of the pixel temperature, thus providing a 16-bit temperaturevalue for each pixel. Other encoding schemes are possible as well. Thetemperature map may be transmitted in addition to, but separately from,the video output from the post-AGC module 214 (e.g., video contrastinformation). Since the temperature map is kept separate, it may not benecessary to trace the temperature measurement accuracy and radiometricaccuracy through the contrast manipulation from the AGC module 214.

The radiometric thermal camera (e.g., radiometric infrared camera)according to one embodiment of the invention may also produce a videosignal or video data that is transmitted with, but separately from, thetemperature map. The video data generated can be post-AGC, or processedby other contrast enhancement techniques, and can have a resolution of320×240 and an 8-bit grayscale, which contains 256 contrast levels. Inone embodiment of the invention, the camera, or external devices (e.g.,smart phone, computer, etc.) that may be in communication with theradiometric thermal camera (e.g., radiometric infrared camera), mayoperate a software application enabled to convert the 8-bit grayscale toa selectable color palette and/or other customizable features determinedby a user. The video data may be transmitted at a frame rate of 9 Hz. Inanother embodiment of the invention, the video data may be transmittedat a frame rate of 30 Hz.

Another advantage achieved by embodiments of the present invention isthat the temperature lookup table enables the conversion of the postNUC/BPR data to directly represent the temperature accurately.Performing this repeatable conversion before AGC processing allows muchgreater flexibility in the methods possible for contrast enhancement ofthe video. It also allows greater control of the bandwidths needed forthe video data separately from the temperature data. Further, the lookuptable may be small relative to the total amount of information processedand transmitted, as the LUT can represent a map from a 14-bit space toan 8-bit space. Although video may be outputted at a first rate (e.g., a9 Hz or 30 Hz rate), the temperature data may be outputted at a lowerrate (e.g., 1 Hz). Typically, thermal data may not be updated at videoframe rates, but can be updated at much slower frame rates. Thus,according to embodiments of the invention, the bandwidth may be managedby slowing down the frame rate that the temperature map is set at, forexample, at 1 Hz.

In another embodiment of the invention, a reduced pixel resolution maybe outputted for the temperature map instead of the full 320×240resolution. Thus, the temperature map output may be decimated down to160×120 or even lower, decreasing the bandwidth even further since thetemperature data remains completely separate from the video contrastdata. Accordingly, in an application used in real time, the temperatureat any place within a scene may be extrapolated from data contained inthe temperature map. Additionally, images of the scene may be stored foruse later and a copy of the temperature map may be stored with theimages of the scene. Images of the scene and the correspondingtemperature maps may be stored for post-processing, for example, todetermine other temperature points within the scene and then determineat the time the image was stored with the full temperature map.

In another embodiment of the invention, encoding techniques performed bythe encoder 222 may be adjustable to include more for the integer andless for the decimal. Other encoding techniques may be implemented andmay have benefits in architectures to add compression to the processingand transmission of the temperature map. Therefore, depending on aparticular application for the video contrast data and temperature data,a specific dynamic range, and/or accuracy requirements, differentencoding methods and varying encoding rates may be implemented.

Thus, according to embodiments of the invention, for example, in onesecond, nine frames of video may be transmitted. For every video frame1/9th of the temperature map can be appended to the video frame. Assuch, a full temperature map may be transmitted in nine frame of video,thus encoding an overall video and temperature map stream. The video andtemperature map streams may be transmitted via an Ethernet interface 226to a wireless interface 228, for example, an interface enabled for Wi-Fi802.11 protocols. Further, the wireless interface 228 may be coupled toa communications interface 230. The Ethernet interface 226 andcommunications interface 230 may further be coupled to a USB typeinterface 232. The USB interface 232 and wireless interface 228 may havebidirectional communications with external devices, such as an externaldisplay (e.g., monitor or television) or processing device (e.g.,personal computer). The USB interface 232 may alternatively be used forcharging the camera and thus may be coupled to a power and batterycharging module 234.

According to an embodiment of the invention, 14-bit data may beoutputted from the A/D converter 206 into the NUC/BPR module 208. Atthat same stage, each pixel can be observed and each 14-bit value can bedetermined and assigned to a temperature of each pixel, which is thenincluded in the data of the temperature map. A lookup table may be usedto map the 14-bit video levels at the NUC/BPR module 208 that convertsthe 14-bit video levels into a corresponding temperature map. In otherembodiments of the invention, other methods may be used to convert frompost-NUC/BPR video levels to temperature. For example, formulas orequations can be implemented and calculated in real-time. The lookuptable may be calibrated previously for 14-bit levels, or any othercorresponding input bit level. Given the 14-bit value, the lookup tableis used to assign a temperature to each pixel to create the temperaturemap. Alternatively, the temperature maps may be implemented furtherdownstream in the process and could be transmitted in addition topost-AGC video data, stored, and thus used in a post-process toreconstruct original temperature data. Thus, the temperature maps mayachieve benefits and advantages of reduced bandwidth in transmissions,improved radiometric accuracy, simplified post-processing andflexibility in the methods used for contrast enhancement.

Methods and systems according to embodiments of the present inventioninclude using the lookup table to build the temperature map prior tocontrast enhancement, and transmitting the temperature map with thepost-AGC video data. Additionally, methods and systems according toanother embodiment of the invention create the temperature map prior tocontrast enhancement, in order to allow maximum flexibility in thecontrast enhancement method. Other embodiments of the invention mayinclude the ability to adjust the data rate and/or resolution of thetemperature map in order to manage bandwidth.

In an embodiment of the invention, the output of the NUC/BPR module 208of FIG. 2 is used to create a full resolution map of the temperature ofthe image before it is processed through the nonlinear AGC process. Thefull resolution map of the temperature of the image is generated by thetemperature map generator 216. So for example, for a detector 204capturing images at a resolution of 320×240, the temperature mapgenerator 216 creates a temperature map with a resolution of 320×240.Every pixel in the temperature map represents the temperature ofwhatever object that pixel is looking at.

Depending on the application of the temperature map, embodiments of theinvention also provide flexibility in altering the resolution of thetemperature map, such that it may be easily combined with the videostream for the camera to provide as an output. The temperature map mayalso be transmitted at a different frame rate, and transmitted with thevideo out. Applications for temperature maps may involve overlayingsymbology showing the temperature of various points or regions in ascene, for which full frame rate updating of the symbology is nottypically needed. Additionally, reducing the frame rate of thetemperature map also aids in managing bandwidth. For example, thethermal camera may output video at a rate of 9 Hz and then output thetemperature map at a rate of 1 Hz, essentially transmitting onetemperature map for every 9 frames of video.

By creating this temperature map from the NUC/BPR data from the NUC/BPRmodule 208 that has not yet gone through the AGC module 214, thetemperature map preserves the temperature information and theradiometric information from the detector 204. Preserving this datapermits significant flexibility in performing AGC or using othercontrast enhancement methods. AGC in general is a non-linear process,and in an embodiment of the invention, may involve sending some mappingor a look up table for each video frame at each post-AGC video frame,which detects all the grayscale values, and converts them back toequivalent object temperatures. However, as the AGC 214 processes frameto frame, the mapping may change. Thus, in an embodiment of theinvention, the method may include outputting with each frame, a mappingto relate back to temperatures, which may be kept separate. By havingthe separate temperature map, the system can keep track of temperaturedata separate from video gray levels, which enables more flexibility incontrast enhancements.

In an embodiment of the invention, the temperature map generator 216 maygenerate a temperature map using temperature data extracted from the ADC206. Then the lookup table may map the average response of the arrays oftemperatures and apply those to corresponding pixels in the video frame.However, because the temperature map is generated before non-uniformitycorrection and bad pixel replacement, there may be pixel to pixelvariability, so the average response to the array aids in compensatingfor the individual pixel variability and provides accuracy despite thenon-uniform pixels.

FIG. 3 illustrates a flow chart of an example method 300 according toone embodiment of the invention. In step 310, a radiometric thermalcamera (e.g., radiometric infrared camera) according to embodiments ofthe invention may be configured to receive thermal data. The thermaldata may be converted, in step 312, from analog to digital using ananalog-to-digital converter. In step 314, the digital output from theanalog-to-digital converter may be processed using non-uniformitycorrection and/or bad pixel replacement. The result of thenon-uniformity correction may then be used to generate a two-dimensionalmatrix representing a temperature map using a lookup table to mapdigital values of each pixel to a temperature of a scene, as seen instep 316.

Simultaneously or concurrently in step 317, the result from thenon-uniformity and bad pixel replacement may then be processed to filterout temporal and spatial noise, and to manage the bandwidth. Filteringout temporal noise may be performed by frame integration (executed byframe integrator 210 of FIG. 2). The spatial noise may be filtered outof the video data by the spatial noise filter 211 of FIG. 2. To managethe bandwidth, a throttle 212 of FIG. 2 may adjust the frequency to alower frequency than the data captured from the scene. At step 318, thegain and level of the video from the NUC/BPR module is adjusted by theAGC. The output from the AGC may then be processed through a multiplexerand encoded with the temperature map in steps 320 to be transmitted in adata channel.

After the NUC and BPR processing 314, the temperature map is generatedat 316 concurrently (e.g., simultaneously) as frame throttling, thespatial filtering, temporal filtering, and AGC processing at 318. At320, the output from the AGC at 318 may be processed using multiplexor.As shown in FIG. 2, the camera serial interface receives two datastreams, the temperature maps from 316 and the output of the multiplexorfrom 318.

The video data may be streamed with the temperature map in the serialinterface be creating for each 320×240 frame of video, 1/9th of thetemperature map. Thus, after 9 sequential frames of video aretransmitted, a full temperature map may be reconstructed. For each videoframe, a superframe of data is created; thus for 9 Hz video and 1 Hztemperature map, the superframe would comprise a 320×240 frame of videofollowed by 1/9 of the temperature map, which would be the first 1/9 ofthe 204 rows of the temperature map. The next superframe would comprisethe next 320×240 frame of video with the second 1/9 of the temperaturemap.

In an embodiment, an end user having a mobile device may operate anapplication on the mobile device that is enabled to reconstruct thefractions of the temperature map into a full temperature map. Theapplication operated on the mobile device may receive the superframesand have code programmed to parse the superframe out, separate the videoframe from the partial temperature map, and reconstruct the fulltemperature map and the video stream.

As such, in an embodiment of the invention, the thermal camera'swireless interface may be enabled to communicate with an application ona mobile device of an end user. The application may be capable ofparsing out the temperature map and the video data, and reconstructingthe temperature map and the video data in a format and manner usable anddisplayable to the end user.

According to an embodiment of the invention, the thermal camera haslocal buffering capability sufficient for processing the video andthermal data for transmission to a mobile device. Most mobile devicestend to have their own data storage, as well an ability to include videoand/or temperature maps reconstructed in the application into e-mail,text, and/or social media platforms. However, in another embodiment ofthe invention, the thermal camera may be enabled to have external datastorage, such as an SD card, flash memory, or compact memory.

The temperature map includes object temperatures, specificallyobject-apparent black body temperatures encoded in Kelvin in an 11.5format, where the 11 bits of the 16 bit format, output in an 11.5decoding scheme. In an 11.5 decoding scheme, the first 11 bits are themost significant bits, and represent the integer portion of the Kelvinapparent black body temperature. The last 5 least significant bits arethe decimal portion of that apparent black body temperature in Kelvin.The lookup table are used to take the output of that NUC/BPR module, theoutput containing 0 to 16383 values, and map those values to theirappropriate 16 bit output temperature values. Accordingly, the lookuptable converts the video levels to 16 bit temperatures in Kelvin.

Various encoding schemes may be used. For example, increasing that bitdepth for greater precision or greater dynamic range, or decreasing thatdepth to manage bandwidth if lower dynamic range or lesser precision areacceptable. The temperature map may also be encoded in Celsius,Fahrenheit, or any other temperatures or scale.

The lookup tables used may be factory calibrated or predetermined, andmay be programmed into the thermal camera. In an embodiment of theinvention, the lookup tables used may be given a dynamic range. Forinstance, a dynamic range may be −20 C to 120 C. Another dynamic rangemay be 0 C to 650 C. For any one of those dynamic ranges, the samelookup table may be used for every thermal camera.

Embodiments of the invention provide a method that conveys thetemperature information of a scene derived from the same data that thevideo data is captured and derived from. Further, embodiments of theinvention allow for modification of the video data without impacting thetemperature data, which is achieved by extracting temperature data earlyin the data stream before video processing such that both video andtemperature data presented to an end user downstream can be veryaccurate and easily displayed to the end user.

The thermal camera according to an embodiment of the invention cangenerate a temperature map based on the extracted temperature data andthen transmit the temperature map and the thermal data in variouscommunications channels (e.g., USB). The temperature map is generatedbefore the contrast enhancement, and simultaneously or concurrentlytransmitted with the video—the gray stage color video with thetemperature data. The end user can infer what the temperature of objectswithin the scenes are based on what their grayscale or color is. Methodsaccording to embodiments of the invention allow preservation of all ofthe accuracies in the temperature measurement and radiated accuracy,without being degraded or destroyed by contrast enhancement processesused to display that imagery. The temperature data is kept separately sothat guessing or inferring what the temperature is based on the grayscale values or the color values.

It should be appreciated that the specific steps illustrated in FIG. 3provide a particular method of processing and transmitting temperaturedata of a scene in a temperature map. Other sequences of steps may alsobe performed according to alternative embodiments. For example,alternative embodiments of the present invention may perform the stepsoutlined above in a different order. Moreover, the individual stepsillustrated in FIG. 3 may include multiple sub-steps that may beperformed in various sequences as appropriate to the individual step.Furthermore, additional steps may be added or removed depending on theparticular applications. One of ordinary skill in the art wouldrecognize many variations, modifications, and alternatives.

FIG. 4 illustrates an exemplary system 400 enabled to execute thefunctions and processes described above. The system may comprise aprocessing module 412, such as a central processing unit, or othercomputing module for processing data. The system 400 may include anon-transitory computer-readable medium, such as a static or dynamicmemory (e.g., read-access memory, or the like), storing code forexecuting tasks and processes described herein.

For example, the computer-readable medium may comprise a temperature mapgeneration module 414 to access a lookup table (LUT) database 420,storing lookup tables to map pixel data to temperature data, asrepresented by steps shown in 314 and 316 in FIG. 3. Thecomputer-readable medium may also include an encoding module 416 toencode the generated temperature map with processed video data, as shownin steps 318 and 320 of FIG. 3, for example. The encoding of thetemperature map with the video data may be transmitted to a camerainterface 440 through an I/O module 422, or other suitable userinterface module.

According to an embodiment of the invention, the thermal camera haslocal buffering capability in the data processor 412 that is sufficientfor processing the video and thermal data for transmission to a mobiledevice through the I/O module 422. Most mobile devices tend to havetheir own data storage (mobile device is not shown, but user interface440 of mobile device is shown), as well an ability to include videoand/or temperature maps reconstructed in the application into e-mail,text, and/or social media platforms. However, in another embodiment ofthe invention, the thermal system 400 may be enabled to have externaldata storage, such as an SD card, flash memory, or compact memory.

The temperature map generated by the temperature map generation module414 may include object temperatures, specifically object-apparent blackbody temperatures encoded in Kelvin in an 11.5 format, where the 11 bitsof the 16 bit format, output in an 11.5 decoding scheme, which may bedecoded/encoded by the encoding module 416. In an 11.5 decoding scheme,the first 11 bits are the most significant bits, and represent theinteger portion of the Kelvin apparent black body temperature. The last5 least significant bits are the decimal portion of that apparent blackbody temperature in Kelvin. The lookup tables stored in the LUT database420 are used to take the output of that NUC/BPR module, the outputcontaining 0 to 16383 values, and map those values to their appropriate16 bit output temperature values. Accordingly, the lookup table convertsthe video levels to 16 bit temperatures in Kelvin.

Various encoding schemes may be used by the encoding module 416. Forexample, increasing that bit depth for greater precision or greaterdynamic range, or decreasing that depth to manage bandwidth if lowerdynamic range or lesser precision are acceptable. The temperature mapmay also be encoded in Celsius, Fahrenheit, or any other temperatures orscale.

The lookup tables stored in the LUT database 420 may be factorycalibrated or predetermined, and may be programmed into the thermalcamera. In an embodiment of the invention, the lookup tables used may begiven a dynamic range. For instance, a dynamic range may be −20 C to 120C. Another dynamic range may be 0 C to 650 C. Calibration for theNUC/BPR processing may adhere to lookup tables stored in the LUTdatabase 420.

Embodiments may be practiced with various computer system configurationssuch as infrared cameras, hand-held devices, microprocessor systems,microprocessor-based or programmable user electronics, minicomputers,mainframe computers and the like. The embodiments can also be practicedin distributed computing environments where tasks are performed byremote processing devices that are linked through a wire-based orwireless network. FIG. 4 shows one example of a data processing system,such as data processing system 400, which may be used with the presentdescribed embodiments. Note that while FIG. 4 illustrates variouscomponents of a data processing system, it is not intended to representany particular architecture or manner of interconnecting the componentsas such details are not germane to the techniques described herein. Itwill also be appreciated that network computers and other dataprocessing systems which have fewer components or perhaps morecomponents may also be used. The data processing system of FIG. 4 may,for example, a personal computer (PC), workstation, tablet, smartphoneor other hand-held wireless device, or any device having similarfunctionality.

For example, the system can include a system bus which is coupled to amicroprocessor, a Read-Only Memory (ROM), a volatile Random AccessMemory (RAM), as well as other nonvolatile memory. The microprocessorcan be coupled to a cache memory. System bus can be adapted tointerconnect these various components together and also interconnectcomponents to a display controller and display device, and to peripheraldevices such as input/output (“I/O”) devices. Types of I/O devices caninclude keyboards, modems, network interfaces, printers, scanners, videocameras, or other devices well known in the art. Typically, I/O devicesare coupled to the system bus through I/O controllers. In one embodimentthe I/O controller may include a Universal Serial Bus (“USB”) adapterfor controlling USB peripherals or other type of bus adapter.

RAM can be implemented as dynamic RAM (“DRAM”) which requires powercontinually in order to refresh or maintain the data in the memory. Theother nonvolatile memory can be a magnetic hard drive, magnetic opticaldrive, optical drive, DVD RAM, or other type of memory system thatmaintains data after power is removed from the system. While FIG. 4shows that nonvolatile memory as a local device coupled with the rest ofthe components in the data processing system, it will be appreciated byskilled artisans that the described techniques may use a nonvolatilememory remote from the system, such as a network storage device coupledwith the data processing system through a network interface such as amodem or Ethernet interface (not shown).

With these embodiments in mind, it will be apparent from thisdescription that aspects of the described techniques may be embodied, atleast in part, in software, hardware, firmware, or any combinationthereof. It should also be understood that embodiments can employvarious computer-implemented functions involving data stored in a dataprocessing system. That is, the techniques may be carried out in acomputer or other data processing system in response executing sequencesof instructions stored in memory. In various embodiments, hardwiredcircuitry may be used independently, or in combination with softwareinstructions, to implement these techniques. For instance, the describedfunctionality may be performed by specific hardware componentscontaining hardwired logic for performing operations, or by anycombination of custom hardware components and programmed computercomponents. The techniques described herein are not limited to anyspecific combination of hardware circuitry and software.

Embodiments herein may also be in the form of computer code stored on acomputer-readable medium. Computer-readable media can also be adapted tostore computer instructions, which when executed by a computer or otherdata processing system, such as data processing system 400, are adaptedto cause the system to perform operations according to the techniquesdescribed herein. Computer-readable media can include any mechanism thatstores information in a form accessible by a data processing device suchas a computer, network device, tablet, smartphone, or any device havingsimilar functionality. Examples of computer-readable media include anytype of tangible article of manufacture capable of storing informationthereon such as a hard drive, floppy disk, DVD, CD-ROM, magnetic-opticaldisk, ROM, RAM, EPROM, EEPROM, flash memory and equivalents thereto, amagnetic or optical card, or any type of media suitable for storingelectronic data. Computer-readable media can also be distributed over anetwork-coupled computer system, which can be stored or executed in adistributed fashion.

It is also understood that the examples and embodiments described hereinare for illustrative purposes only and that various modifications orchanges in light thereof will be suggested to persons skilled in the artand are to be included within the spirit and purview of this applicationand scope of the appended claims.

What is claimed is:
 1. A method of generating a temperature map of ascene, the method comprising: receiving, at a detector, light data ofthe scene, wherein the light data includes a plurality of frames ofanalog infrared light data; converting each of the plurality of framesof analog infrared light data from analog to digital format to createdigital infrared light data including a plurality of digital framescorresponding to the plurality of frames of analog infrared light data;creating a lookup table for each frame of the plurality of digitalframes based on the digital infrared light data; generating thetemperature map using the lookup table; separately transmitting thetemperature map and the digital infrared light data in a data channel;integrating the plurality of digital frames in the digital infraredlight data at a frame rate, thereby creating integrated digital videodata; adjusting a gain of the integrated digital video data at the framerate using an automatic gain control to generate an adjusted digitalvideo data; selecting a video output from at least two of the adjusteddigital video data, the digital infrared light data, or the integrateddigital video data to create a multiplexed video output; and separatelytransmitting the temperature map and the multiplexed video output in thedata channel.
 2. The method of claim 1 further comprising: receiving aplurality of frames of non-thermal light data; converting the pluralityof frames of non-thermal light data from analog to digital format tocreate a plurality of frames of digital non-thermal light data; andtransmitting the plurality of frames of digital non-thermal light datain the data channel.
 3. The method of claim 2 wherein non-thermal lightdata includes visible light data.
 4. The method of claim 1 furthercomprising calibrating the lookup table using non-uniformity correctionon the digital infrared light data.
 5. The method of claim 1 furthercomprising: detecting light data of the scene at a first frame rate,wherein a second frame rate of the integrated digital video data islower than the first frame rate; and transmitting the temperature map ata third frame rate, wherein the third frame rate is lower than thesecond frame rate, wherein the temperature map is transmitted at thethird frame rate separately from the digital infrared light data in thedata channel.
 6. An apparatus for generating a temperature map of ascene, the apparatus comprising: a detector configured to receive lightdata of the scene, wherein the light data includes analog infrared lightdata, and includes a plurality of frames; an analog to digital converterconfigured to convert the light data to digital light data, the digitallight data including digital infrared light data; a lookup tablegenerator configured to create and calibrate a lookup table for eachframe of the plurality of frames based on the digital infrared lightdata; a temperature map generator configured to generate the temperaturemap using the lookup table; a data channel interface configured totransmit the temperature map separately from the digital infrared lightdata; a frame integrator configured to integrate the plurality of framesin the digital infrared light data to a frame rate, thereby creating anintegrated digital video data; an automatic gain control unit configuredto adjust a gain of the integrated digital video data at the frame rateto generate an adjusted digital video data; and a controller configuredto select at least two of the adjusted digital video data, the digitalinfrared light data, or the integrated digital video data to create amultiplexed video output, wherein the data channel interface isconfigured to transmit the temperature map separately from themultiplexed video output.
 7. The apparatus of claim 6 wherein thedetector detects the light data of the scene at a first frame rate andwherein the apparatus further comprises: a throttle configured toregulate a second frame rate of the integrated digital video data,wherein the second frame rate is lower than the first frame rate,wherein the data channel interface is configured to transmit thetemperature map at a third frame rate, wherein the third frame rate islower than the second frame rate of the multiplexed video output,wherein the temperature map is transmitted separately from themultiplexed video output in the data channel interface.
 8. The apparatusof claim 6 further comprising a calibrator configured to calibrate thelookup table using non-uniformity correction on the digital light data.9. The apparatus of claim 6 further comprising a calibrator configuredto calibrate the lookup table using bad pixel replacement on the digitallight data.
 10. The apparatus of claim 6 wherein the data channelinterface is further configured to: encode the temperature map and thedigital infrared light data in a data stream; and communicate with awireless communication network to receive and transmit the data streamto one or more external devices.